1. Field of the Invention
This invention relates to opticalneural networks.
2. Description of the Related Art
A neural network is a computational structure modeled on biological processes. The models are composed of many nonlinear computational elements operating in parallel and arranged in layers analogous to biological neural nets. Computational elements or nodes are connected via weights that are typically adapted during use to improve performance. The output of each neuron may be connected to many other neurons via connection weights and is a nonlinear function of the weighted sum of the outputs of those connected neurons; the nonlinear function is usually called the neuronactivation function. Neural networks have particular potential in areas such as speech and pattern recognition and categorization, compensation of focal plane array nonuniformities and combinationally complex scheduling problems, in which many hypotheses are pursued in parallel, high computation rates are required, and currently available systems do not equal human performance.
Instead of performing a program of instructions sequentially, as in a typical computer, neural networks explore many competing hypotheses simultaneously using massively parallel networks composed of many computational elements connected by links with variable weights. Neural networks can be "taught" to produce desired outputs in response to given inputs by an iterative sequence of adjustments. A general introduction to the subject is provided in Lippmann, "An Introduction to Computing with Neural Nets", IEEE ASSP Magazine, Apr. 1987, pages 4-22.
A neural network is "trained" to produce an appropriate response to a class of inputs by being presented with a sufficient number of examples during the learning phase. The presentation of these examples causes the strength of the connections between the neurons that compose the network to be modified according to the specifics of the learning algorithm. A successful learning procedure results in a trained network that responds correctly when it is presented with the examples it has seen previously, and also with other inputs that are similar to the known patterns.
Many neural network models have been described in the literature for such processing tasks as associative memory, unsupervised pattern categorization, and pattern recognition, as well as others. The models differ in the degree of connectivity and the equations for adjusting the connection weights in response to input patterns ("learning"). Most of the models use some form of "Hebbian" learning in which the incremental change in a weight connecting two neurons is given by the product of the neurons' activation levels.
Investigations have been conducted recently into the implementation of neural networks by optical means. The massive interconnectivity, parallelism and analog nature of optical architectures are good matches to the requirements of neural network models. In a related series of investigations involving associative memories, phase conjugate mirrors (PCMs) were employed in a holographic memory system in which pairs of information patterns were stored in such a way that the introduction of one pattern results in the recall of another. Such associative memories, however, can recognize only specific images, as opposed to a more generalized neural network which can be taught to recognize patterns belonging to a general class, e.g. generalize from examples. The associative memory work is exemplified in Patent Nos. 4,739,496, 4,750,153 and 4,762,397, all assigned to Hughes Aircraft Company, the assignee of the present invention.
Previous optical holographic implementations of neural network models used a single grating in a photorefractive crystal to store a connection weight between two neurons; each pixel in the input/output planes corresponded to a single neuron. An interference was established between two beams to form a sinusoidal grating, which thereafter induced a phase shift in a readout beam via the photorefractive effect. The weighting of the connection between each pair of neurons was encoded in the modulation depth of a single associated grating. Optical holographic approaches are discussed in the following: Psaltis, Yu, Gu and Lee, "Optical Neural Nets Implemented with Volume Holograms", Lake Tahoe Meeting 1987, pages 129-132; Psaltis, Brady and Wagner, "Adaptive Optical Networks Using Photorefractive Crystals", Applied Optics, Vol. 27, No. 9, 1 May 1988, pages 1752-1759; Wagner and Psaltis, "Multi-Layer Optical Learning Networks", Applied Optios, Vol. 26, No. 23, 1 December 1987, pages 5061-5076; Owechko, "Optoelectronic Resonator Neural Networks", Applied Optics, Vol. 26, No. 23, 1 Dec. 1987, pages 5104-5110; and Owechko, Soffer and Dunning, "Optoelectronic Neural Networks Based on Holographically Interconnected Image Processors", SPIE Vol. 882, 1988, pages 143-153.
The latter two articles describe an opto-electronic nonlinear holographic associative memory consisting of a hologram situated in a phase conjugate resonator formed by four-wave mixing. A PCM separate from the hologram storage mechanism provides phase conjugation and optical routing.
The disclosed use of a single grating in a photorefractive phase conjugate crystal to store a connection weight between two neurons relies upon the Bragg condition for angularly selective diffraction from a grating to avoid cross-talk between neurons. However, because of the angular degeneracy of the Bragg condition, neuron isolation is incomplete and cross-talk results which would prevent proper operation of the neural network. To remove the cross-talk, the neurons must be arranged in special patterns in the input/output planes. This results in an under-utilization of the total available output from the spatial light modulators (SLMs) used to present the input image, and incomplete utilization of the SLMs. In particular, if the SLMs are capable of displaying N.times.pixels, the disclosed methods can store only N.sup.3/2 neurons and N.sup.3 interconnections. This compares with an optimal storage capacity of N.sup.2 neurons and N.sup.4 interconnections if the entire capacity of the SLMs were used. Since N is typically in the range of 500-1,000, this under-utilization of the SLM capacity is substantial. Furthermore, the prior holographic systems use a lithium niobate medium to store the holograms, resulting in a bulky system that is difficult to work with.